Detecting high-order epistasis in nonlinear

7 High-order epistasis has been observed in many genotype-phenotype maps. These multi-way inter8 actions could have profound implications for evolution and may be useful for dissecting complex 9 traits. Previous analyses have assumed a linear genotype-phenotype map, and then applied a linear 10 high-order epistasis model to dissect epistasis. The assumption of linearity has not been tested 11 in most of these data sets. Using simulations, we demonstrate that neglecting nonlinearity leads 12 to spurious high-order epistasis. We find we can account for this nonlinearity in simulated maps 13 using a power transform. We then measure and account for nonlinearity in experimental maps 14 for which high-order epistasis has been previously reported. When applied to seven experimen15 tal genotype-phenotype maps, we find that five of the seven exhibited nonlinearity. Correcting 16 for this nonlinearity had a large effect on the magnitudes and signs of the estimated high-order 17 epistatic coefficients, but only a minor effect on additive and pairwise epistatic coefficients. Even 18 after accounting for nonlinearity, we found statistically significant fourth-order epistasis in every 19 map studied. One map even exhibited fifth-order epistasis. The contributions of high-order epis20 tasis to the total variation in the map ranged from 2.2% to 31.0%, with an average across maps 21 of 12.7%. Our work describes a simple method to account for nonlinearity in binary genotype22 phenotype maps. Further, it provides strong evidence for extensive high-order epistasis, even after 23 nonlinearity is taken into account. 24 2 . CC-BY 4.0 International license peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/072256 doi: bioRxiv preprint first posted online Aug. 30, 2016;